Scalable Representation Structures for Visuo-Spatial Reasoning - Dynamic Explorations into Knowledge Types

A sizable fraction of current research into human visuospatial knowledge processing explicitly or implicitly suggests a spatial processing of certain knowledge types and a visual processing of others. Similarly, many formal and technical approaches for representing and processing visuospatial information in artificial intelligence, in computational cognitive modeling, or in knowledge representation and reasoning explicitly or implicitly treat visual and spatial information as belonging to separate types. While there exists good evidence for some differences in mental processing of different visuo-spatial knowledge types, there is much less reason to maintain the currently ascribed separation between the visual and the spatial. We provide arguments on why strict dichotomies seem unwarranted with regard to descriptions of human mental spatial reasoning and disadvantageous for the formal and technical approaches. We build upon a synopsis of psychological evidence for the existence of multiple knowledge type specific representations in human visuo-spatial reasoning and discuss the notion of scalable representation structures. In absence of proof to the contrary, it seems better practice to assume that (a) many of the type differences attributed to visuo-spatial knowledge processing are gradual rather than qualitative in nature, and that (b) tasks involving visuo-spatial knowledge of several types are often mentally processed through dynamic associations of structures for processing basal knowledge types. The paper calls for more investigations of human reasoning in visuo-spatial tasks in which knowledge types dynamically change during reasoning. It outlines a research framework for systematically investigating different basal visuospatial knowledge types and their combinations with regard to cognitive and computational plausibility. Current research is related to the framework, including research on Casimir, our computational cognitive architecture for reasoning with visuo-spatial knowledge. We argue that a more systematic course of research along the lines of the proposed framework will not only lead to more appropriate descriptions of human cognition (regarding visuo-spatial knowledge processing) but may also spawn more integrated and versatile forCopyright c © 2009, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. mal and technical approaches for dealing with visuo-spatial information. The Visual and the Spatial In this article, we will address a few issues that concern our current understanding of knowledge types frequently described to be involved in mental reasoning with visuospatial knowledge. In particular, we will focus on the interrelations between knowledge types, such as between different spatial types (e.g., holding knowledge about directions, distances, topology) as well as between these and types often described as visual. Second, we will suggest a novel research framework which we believe to be quite capable of bringing about a much more accurate understanding of which knowledge types are actually involved in mental visuo-spatial reasoning and under which conditions, and of how visuo-spatial knowledge gets dynamically maintained and integrated within and across types during reasoning. Various past and ongoing research addressed and addresses questions related to sensory modality in reasoning with visuo-spatial knowledge. For example, are the involved mental representation structures more adequately described as visual, spatial, motor-related, etc., as propositional, or as abstract and non-modal? What is the involvement of visual or motor imagery processes in mental reasoning with visuospatial knowledge, and which subsystems of working memory are drawn upon under which circumstances? A rough grouping of existing research identifies two main camps: one includes modalist accounts which describe the involved mental representations, at least partially, in terms of mental images (Bertel et al. 2006; Kosslyn 1994), through analogies drawn from diagrams (Chandrasekaran et al. 2004), and as tapping into and relying on mental visuo-spatial subsystems (Kosslyn and Thompson 2003; Ishai and Sagi 1997). This group further involves motorbased accounts that, for instance, explain aspects of visuospatial reasoning through enacted or reenacted shifts of eye fixations or visual attention (Richardson and Spivey 2000; Laeng and Teodorescu 2002; Johansson, Holsanova, and Holmqvist 2005) or draw connections between success in reasoning and an embodying of the solution in terms of movements or attention shifts (Thomas and Lleras 2009). The second group includes non-modal accounts which are, among others, rooted in analogies to propositions (Jahn 2 Multirepresentational Architectures for Human-Level Intelligence: Papers from the AAAI Fall Symposium (FS-09-05)